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Abstract 

In this paper, redundant random matrix ensembles (abbre- 
viated as redundant random ensembles) are defined and their 
stopping set (SS) weight distributions are analyzed. A redun- 
dant random ensemble consists of a set of binary matrices 
with linearly dependent rows. These linearly dependent rows 
(redundant rows) significantly reduce the number of stopping 
sets of small size. An upper and lower bound on the average 
SS weight distribution of the redundant random ensembles are 
shown. From these bounds, the trade-off between the number 
of redundant rows (corresponding to decoding complexity of 
BP on BEC) and the critical exponent of the asymptotic growth 
rate of SS weight distribution (corresponding to decoding 
performance) can be derived. It is shown that, in some cases, 
a dense matrix with linearly dependent rows yields asymptot- 
ically (i.e., in the regime of small erasure probability) better 
performance than regular LDPC matrices with comparable 
parameters. 
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I. Introduction 

On binary erasure channel (BEC), the decoding performance 
of belief propagation (BP)-based iterative decoder of low- 
density parity-check(LDPC) codes is dominated by combina- 
torial structures in a Tanner graph, which are called stopping 
sets (SS)[1]. Di et al.[l] introduced the idea of stopping sets 
and presented a recursive method to evaluate the average block 
and bit error probabilities of LDPC codes [8] of finite length 
on BEC[1]. Orlitsky et al. [2] found the asymptotic behavior 
of the SS weight distributions of bipartite graph ensembles 
and extended the results of Di et al. to the irregular code case. 

For a given binary linear code C, it is hoped to find the best 
representation of C (i.e., a parity check matrix) which yields 
the smallest block (or bit) error probability when it is decoded 
with iterative decoding on BEC. A parity check matrix which 
defines C can be a redundant parity check matrix, which is 
not a full-rank matrix: that is, it can contain some linearly 
dependent rows. For example, some finite geometry LDPC 
codes require a redundant parity check matrix to achieve good 
decoding performance with BP. Recent works of Schwartz 
and Vardy[3], Abdel-Ghaffar and Weber[4], Hollmann and 
Tolhuizen[5] indicate that the stopping set weight distribution 
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of a given matrix can be improved by appending linearly 
dependent rows to the original matrix. 

Recent developments described in studies of the aver- 
age weight distributions of LDPC codes, such as Litsyn 
and Shevelev[9][10], Burshtein and Miller[ll] Richardson 
and Urbanke[6], imply that ensemble analysis is a powerful 
method for investigating typical properties of codes and ma- 
trices, properties are not easy to obtain from an instance. Fur- 
thermore, from the asymptotic behavior of typical properties 
such as these, we often can predict a threshold phenomenon. 

The average stopping set weight distributions presented in 
[1] and [2] are a useful decoding performance measure (for BP 
on BEC) of a given ensemble of parity check matrices. The 
distribution can be used for optimizing an ensemble suitable 
for BEC. BEC is not only of practical interest, but also can be 
considered as a good starting point for theoretical studies of 
performance analysis of BP for more general channels, such 
as binary input symmetric output channels [6]. 

In this paper, redundant random matrix ensembles (abbre- 
viated as redundant random ensembles) are defined and their 
SS weight distributions are analyzed. The redundant random 
ensemble consists of a set of binary matrices with linearly 
dependent rows. These linearly dependent rows (redundant 
rows) significantly reduce the number of stopping sets of small 
size. An upper bound and a lower bound on the average SS 
weight distribution of redundant random ensemble will be 
shown. From these bounds, the trade-off between the number 
of redundant rows (corresponding to decoding complexity 
of BP) and the critical exponent of the asymptotic growth 
rate of SS weight distribution (corresponding to decoding 
performance) can be derived. 

II. Average SS weight distribution 

In this section, some notation and definitions required in 
the paper are introduced. Furthermore, some known results on 
average SS weight distributions are briefly reviewed. 



A. Stopping set and SS weight distribution 

Let F 2 be the binary Galois field with elements {0, 1}. The 
operator o denotes the integer ring inner product defined by 



hox 



A 



-h 2 X 2 - 



h\X\ - 

Fg and x = (xi, x%, . 



■ + h n x n for h = (h 1 ,h 2 ,. ■ ■ , h N ) e 
.,x n ) € F% (n > 1). The additions 



in the above definition of o is the addition of the integer ring 
(i.e., 1 + 1 = 2). In this paper, the addition of F 2 is denoted 
by © (i.e., 1 © 1 = 0). 
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For a given x G and an to x n binary matrix H (m, n > 
1), the 55 indicator qii(x) is defined by 



qu{x) = #{i G [1, to] :h t ox = 1}, 



(1) 



where hi denotes the i-th row vector of H, and we denote 
the cardinality of a given finite set X by #X. The notation 
[a, b] means the set of consecutive integers from a to b. The 
stopping set is defined as follows: 

Definition 1 (Stopping set): If qn(x) — then x € i 7 ^ is 
called a 55 vector of ii". The support set of x, 



A 



(2) 



A 



Sa; = {« £ [l,n] : ajj = 1}, 

is called a stopping set of _hQ. □ 
Note that if there exists a row vector hi(i G satis- 
fying hi o x = 1 then x is not an SS vector. Let y 
(j/i, 2/2, ■ ■ ■ j Vn) 6 {0,1, e}™ be a received word through a 
BEC, where e denotes the erasure symbol. It is known that 
BP fails to decode y if and only if the erasure support set 

Ey = {i G [l,n] : yi — e} contains a non-empty stopping 
set of H. This property justifies the study of SSs in order to 
reveal the BP decoding performance for BEC. 

The next definition provides the definition of the SS weight 
distribution and the stopping distance: 

Definition 2 (SS weight distribution and stopping distance): 
For a given to x n(m,n > 1) matrix H, the SS weight 
distribution {S' ti ,(ff)}™ =0 is defined by 



S W (H 



A 



I[q H (x) = 0] 



(3) 



for < w < n, where Z^ n ' w ^ is the set of constant weight 
binary vectors of length n whose Hamming weights are w. 
The notation I[condition] is the indicator function such that 
I[condition] — 1 if condition is true; otherwise, it gives 0. 
The stopping distance of H is defined by 



A 



A(H) = mm{w G : S W (H) ^ 0}. 



Example 1: Let 



H = 



(4) 



□ 



(5) 



A 



In this case, W — {0, {1, 3}, {1, 2, 3}} is the set of stopping 
sets of H and 2[ 1, ™1\M / is the set of non-stopping sets of 
H. The SS weight distribution is given by {S W (H)}%, =0 = 
{1, 0, 1, 1} and the stopping distance is A(i?) = 2. rj 

B. Average 55 weight distribution 

Suppose that Q is a set of binary rn x n matrices (to, n > 1). 
Note that we allow the possibility that Q may contain some 
matrices with the same configuration. Such matrices should 
be distinguished as distinct matrices. We assign the same 

This definition of SS is not exactly the same as the original definition[l]. 
The present definition covers the case where there exists a variable node 
without an edge(7.e., a zero column). 



probability, 1/JfG, to each matrix in Q. Let f(H) be a real- 
valued function which depends on H G Q. The expectation of 
f(H) with respect to the ensemble Q is defined by 

Eg[f{H)] = P(H)f(H) = ^ E /(*0- <9 
Heg " Heg 

The average SS weight distribution is defined as follows: 

Definition 3 (Average 55 weight distribution): The average 
SS weight distribution of a given ensemble Q is defined by 



Si = Eg[S w (H)} 



(7) 



for < w < n. □ 
One of the most important properties of an ensemble is its 
symmetry. Although several types of symmetry are shown to 
be useful in [12], the following simple definition is sufficient 
for the purposes of this paper. 

Definition 4 (Symmetry of an ensemble): If the equality 

#{# G Q : q H {xi) = 0} = #{H G Q : q H (* 2 ) = 0} (8) 

holds for any x\,x^ G Z^ w ^ and any w G [0, n], then the 
ensemble Q is called symmetric. rj 
There is a simple expression of the average SS weight distri- 
bution for a symmetric ensemble. 0. The next lemma shows 
that the evaluation of the average SS weight distribution is 
equivalent to a counting problem of matrices satisfying a 
certain condition. 

Lemma 1: If Q is symmetric, then 



#{H G Q : q H (x % 



0} 



(9) 



holds for w G [0, n]. The vector x w G Z^ n ' w ^ is the binary 
vector whose first w-elements are one, with all other elements 
zero. 

(Proof) The average SS weight distribution of Q can be 
transformed into the following form, 

5g = Eg [S w {H)\ 

= E p w E = o] 

Heg sez(".-) 

= E E P{H)I[q H {x) = 0] 

a;ez(».») Heg 

= -h E #{Heg:q H (x)=0} (10) 



O 



xez< r 



#{H G G : = 0}. 



(11) 



The last equation follows from the assumption; namely 
G Q : qn{x) — 0} takes the same value for any 

x G Z( n ' w \ ' n 

C. Average 55 weight distributions of known ensembles 

In this subsection, the average SS weight distribution of 
three well-known ensembles: the random ensemble, the con- 
stant row weight ensemble and the bipartite ensemble, will be 
shown. 

2 Note that all the ensembles discussed in this paper are symmetric. 
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1) Random ensemble: The random ensemble H m ,n is the (Proof) Substituting the average SS weight distribution (TT2l) 

set of all binary to x n matrices(TO, n > 1). Thus, the size of into expression (flTT l. we have 
H m .n is equal to 2 mn . The following lemma gives the average \ 
SS distribution of the random ensemble. The key of the proof ^ = ^ ^°&2 ^t n 

is to count #{# G 7£ m ,„ : q H (x w ) = 0}. 1 



(l-B)r, 



Lemma 2: The average SS distribution of the random en- 



semble 1Z m;n is given by ^ 



= lim - log 2 ' (1 - ln2- ln )^-^ n 



S™ m,n =('")(!- w2~ w ) m (12) i 



lim - log 2 



lim -log 2 (l -fe2^™)( 1 -^" 



=2 

n — >oo 77, 



for < w < n. = H{£) + (1 - R) Jim log 2 (l - £n2~ 

(Proof) From the definition of the ensemble, it is evident that 
the following equality holds, 



= H{1), (20) 



Note that in deriving the fourth equality from the third equality, 
#{H G Km n : q H {x w ) = 0} = G F 2 " : h,oa; u , ^ the following equality 

< 13 > 1 /n\ 

It is easy to show that the equality - log 2 I I = H(£) + o(l), (21) 

G Fo : h o x w =/= 1} = (2 W — w)2 n ~ w (14) was used, where o(l) denotes terms which converge to in 

the limit as n — > oo. rj 
holds, since 2J Constant row weight ensemble: The constant row weight 

ensemble C m . n . r consists of all the binary m x n matrices 
G F 2 " : hox w = 1} = (15) whose rows have exactly weight r (m,n > l,r > 1). The 

size of the ensemble is, thus, 
Combining the above results, we get /■ 



1 / n 



The average weight distribution of this ensemble was shown 
#{H G K m , n : q H (x w ) = 0} in [9]. 

The following lemma shows the average SS distribution of 



— _ [ ] (2" — w2 n ~ w ) m Cm,n,r- The proof is similar to that of Lemma[2] 

2 mn \wj Lemma 4: The average SS distribution of the constant row 

weight ensemble C m , n ,r is given by 



(l-w2- w ) m . (16) 



n—w 

n 



1-™^ . (22) 



Note that in deriving the second equality from the first equality, \w 

the symmetric property of the random ensemble and Lemma 

□ , for < w < n. 

was used. n _ — _ — , . . 

™ . , . , oo ■■ •, • (Proof) Combining 

The asymptotic growth rate of the average SS distribution 

(for simplicity, abbreviated as the asymptotic growth rate) of ^{H G C m , n ,r '■ qn(x w ) = 0} 

the random ensemble is defined by = e z^ r) :hox w ^ l}) m (23) 



#{heZ^:hox w ^l}= r) -w( n ™), (24) 



A, = lim -log 2 5 ; (1 - fl '"-, (17) 

where R(0 < R < 1) is called design rate and £(0 < i < 1) 
is the normalized weight. The asymptotic growth rate reflects we hare 

the asymptotic (in the limit as n goes to infinity) behavior ('V 75 ^ fn — w 

of the average SS weight distribution for fixed design rate " * ^ m,n,r • Qh(x w ) — ' \\ r J W \r — 1 
and normalized weight. The next lemma gives the asymptotic (25) 
growth rate of random ensembles. The average SS weight distribution is thus given by 

Lemma 3: The asymptotic growth rate of the random en- c 1 
semble is given by " " 5 - m '"'" = 2^ ^ H ^G:q H {x) = 0} 

7T m,n,r £cgZ( „, ro) 



n r )-w [ n ~ w 

w I \\r I \ r — 1 



X e = H{£), 0<£<l, (18) 
where is the binary entropy function defined by 

= ( " ) ( 1 - w-^tstH ■ (26) 
H(a;) = -a ; log 2 a ; -(l-a ; )log 2 (l-^). (19) W (™) / 



\ / fn—w\ 
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In the above, the symmetric property of the ensemble is used 
in deriving the second equality from the first. rj 
The asymptotic growth rate of the constant row weight 
ensemble is defined by 



&(i?,r) = lim -log 2 S^ 

n— >oc Tl 



(27) 



for < R < 1 and < £ < 1. The next lemma gives the 
explicit form of the average growth rate: 

Lemma 5: The asymptotic SS weight distribution of the 
constant row weight ensembles is given by 



Ce(R,r) = H(£)-(l-R)\og 2 



1 



i - r£{i - ey 



~ (28) 



for < R < 1 and < £ < 1. 

(Proof) By using the lower and upper bounds on binomial 
coefficients given by 



fc! 



n" 1 I fc \ I n\ n" 
■ exp I I < I . 1 < 



k ~ fc! ' 



(29) 



we obtain 



in^U > tnr(n-£ny-i exp { (r - 1) 



(") 

= r£(l - I) 7 *' 1 exp 



n — in 
(r- l) 2 
n — in 



(30) 



and 



in 



" _ ™) ^ inr(n — £n) r 



(") 



< 



■exp — 

n 



r£(l - If' 1 exp — 
1 n 



These bounds imply that 

(") 



= rf(l-^) r - 1 +o(l) 



(32) 



since r is constant (/.e. not a function of n). Using this 
equation, we obtain immediately the asymptotic SS weight 
distribution, 



lim Ilog^ 1 -"'"-" 

1 / n\ ( (n-tn'j \ ( 1 -- R )™ 

lim - log 2 I 111- fa ^ 



= H{£) + (1 — R) lim log 2 1 - in 







+ (1 - i?) log 2 (l - r*(l - f)^ 1 ). (33) 



□ 



3) Bipartite ensemble: The bipartite graph ensemble (ab- 
breviated as a bipartite ensemble) B n . c .d is the ensemble of 
regular bipartitegraphs of variable node degree c and check 
node degree d. The following lemma is due to Orlitsky et 
al[2]. 

Lemma 6 (Orlitsky et al): The average SS weight distribu- 
tion of B n . Ct d is given by 



, BiCid _ coef [((1 + x) d - dx)% n , x wc ] 
> ~ ( nc \ 

\wc) 



(34) 



where coef [f(x), x % ] denotes the coefficient of a polynomial 
f(x) corresponding to the term x l . rj 
The asymptotic growth rate of (c, d)-bipartite ensemble is 
defined by 

j e (c,d)= lim - log 2 Sf™' c,d (35) 
n— *oo n 

for < £ < 1. It is shown in [2] that the asymptotic growth 
rate ji(c, d) has the form 

le (c,d) = >g e f ^^ V(c-W), (36) 



where a?o is the only positive solution of 

x((l + a-) d - 1 - 1) 



(37) 



(1 + x) d - dx 

and H e (x) is the entropy function with base e defined by 

H e (x) = -xlog e (x) -(1-x) log e (l - x). (38) 

III. Average SS weight distributions of redundant 

RANDOM ENSEMBLE 

In this section, we discuss the average SS weight dis- 
tributions of extended ensembles obtained from the random 
ensemble. 



(31) A. Redundant extension 

Before commencing a discussion of redundant extensions, 
it is perhaps worthwhile to consider how some stopping sets 
can be eliminated by extending a matrix. 
Example 2: Consider the matrix 



H 



A 




(39) 



It is easy to see that {2, 3, 4} is a stopping set (the sub-matrix 
composed of the second, third and fourth columns has no row 
of weight 1). Appending (0 1) (obtained by adding the 
first and second rows of H) to H as a row vector, we have a 
modified matrix H', 



H' 



A 



110 
10 11 
V 1 / 



(40) 



Strictly speaking, to define the average SS weight distribution of B n c ^, 
we need a graph-based definition of the stopping sets and ensemble average. 
Details can be found in [2]. 
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We can observe that the weight of the last row of the sub- 
matrix corresponding to the second, third and fourth columns 
is 1. This implies that {2,3,4} is no longer a stopping set of 
H' . Note also that the row spaces spanned by H and W are 
exactly the same. rj 

The previous example demonstrates the possibility that the 
SS weight distribution could be improved by adding linearly 
dependent rows (called redundant rows) to a given matri)(0- 

Let H be a binary m x n (1 < m < n) matrix, 



( hl \ 

h 2 



H 



A 



\ h m J 



(41) 



Let L be a positive integer which is a divisor of m. For 1 < 
i < 2 L - 1, < £ < m/L - 1, we define af 1 by 



All linear 
combinations 
(except for zero) 




(2 -1)(m/L) 



,(L) 



A sub-block of H corresponds to a sub-block in (for 
example, sub-block A corresponds to sub-block A'). The row 
vectors in sub-block X' (X £ {A,B,C} ) can be obtained by 
constructing all linear combinations (except for the zero 
combination) of the row vectors in sub-block X. 



Mi)) 



( h Lt+ i ^ 
hu+2 



\ h Le+L J 



Fig. 1. Redundant extension of a parity check matrix. 



(42) 



where bj (i) is the j-th bit of binary representation of i, namely, 
i = J2j=i^b 3 (i).ln other words, a!;p is a linear combination 
of h Le+k (l <k<L). 

The redundant extension of a given matrix is defined as 
follows: 

Definition 5 (Redundant extension): The redundant exten- 
sion of H, denoted by H^ L \ is the matrix whose row vectors 
are af ] for 1 < i < 2 L - 1 and < £ < m/L - 1. In other 
words, H 1 -^ is given by 



H (L) 



h 



(43) 



where h[ 2L _u e+i = a- £) for 1 < i < 2 L - 1 and < I < 
m/L — 1. The parameter L is called extension degree. The 
number of row vectors in is (2 L — l)(m/L). rj 

Figure Q] illustrates the idea of the redundant extension. 

From the above definition of the redundant extension, it is 
clear that the row spaces of H and are the same. In 

other words, the code defined by H coincides with the code 
defined by H^ L \ However, although the codes defined by H 
and > are the same, H and may have different SS 

weight distributions. 

Example 3: For a given mxn matrix H, is expressed 



4 It is evident, from the definition of SS, that the addition of redundant rows 
does not introduce a new SS which is a non-SS of the original matrix. 



as 



hi © h 2 

h 3 

h A 
h 3 © h 4 



h, 



-1 © K, 



(44) 



□ 

The definition of the redundant extension of a matrix 
naturally leads to the following definition of the redundant 
extension of a given ensemble: 

Definition 6 (Extended ensemble): Consider the case where 
a random ensemble Q which consists of mxn binary matrices 
is given. Let L be a divisor of m. The extended ensemble of 
Q, denoted by Q^ L \ is defined by 



:(£ 



{H^ L) :H eG}. 



(45) 



The size of the ensemble #C/^ L ' is equal to the size of the 
original ensemble An equal probability is assigned to 
each matrix in Q^ L >. rj 
The redundant random ensemble which is the main subject of 
this paper is the extended ensemble of a random ensemble, 
which is denoted by 1Z 



(L) 



B. Redundant random ensemble: L = 2 

In this subsection, we discuss the average SS weight dis- 

(2) 

tribution of the redundant random ensemble TZ m . n . In this 
case, we can derive a simple exact formula for the average 
SS weight distribution. 
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(2) 

Suppose that the first 3-rows of H e 




Taking the case (a, (3) = (0, 1) into consideration, we imme- 
diately have 



(46) 



7=2 



(w - 7). 



(57) 



are given. Our first task is to count the number of pairs Substituting this equation into Eq.(E3, we obtain the claim of 
(hi, h 2 ) G F 2 n x F£ satisfying qg(x w ) — 0. Let us define U $i e lemma 
by 



□ 



A 



U = #{(hi, h 2 ) E F? x F™ : hi o x w ^l,h 2 o x w ^ 1, 



The following theorem is an immediate consequence of 
Lemma [7] 

Theorem 1 (Average SS distribution of lZm]n )■' The aver- 



(hi © h 2 ) o x ^ 1}. (47) a ge SS weight distribution of 1l%] n is given by 



(2) 

Note that to/2 sub-blocks exist in H £ H n 'm, and these sub 



= A-i (( 2 - 



w2 r 



) 2 -v) 



m/2 



blocks can be chosen independently when we count G 

f2} 

W.,m : Qh{x w ) = 0}. This observation leads to the following for 1 < m < n, w > 1, where V is defined by 
equality, 



m/2 



#{HeK^ :q H (x w ) = 0} = U 



The next lemma gives a simple description of U. 
Lemma 7: For 1 < m < n, w > 1, U is given by 



(48) 



y A 22(n-io)+l 



E 

7=2 



(l0-7). 



(58) 



(59) 



(2) 

(Proof) The average SS weight distribution of TV n . m can be 
derived in the following way: 



22(n-tu) + l 

7=2 



(w - 7). (49) 



17 = (2" - W 2™-™) 5 
(Proof) Let 

y = #{(/n, h 2 ) G F? x F 2 " :h 1 ox w ^l,h 2 o x w ± 1, 

(hi @h 2 )ox w = 1}. (50) 



1 



^{ffeT^ = 0} 



yL#{Heni%:q H (x w )=0} 



Using the equality 

#{(hi, h 2 ) G F 2 " x F 2 " : hi o a;^ ^ 1, h 2 o ^ 1} 

= (2 n -w 



_jjm/2 



= W. (( 2 « _ u ,2 n - tu ) 2 - y) 



i/2 



(60) 



we have 



The second equality follows from the symmetric property of 
2n-w\2 ^j-j the ensemble, while the third equality is derived from Eq.j48l). 

The last equality is due to Lemma [7] rj 
Example 4: Consider the case where n = 4, to = 2. The 



U = {{h 1 ,h 2 )eF?xF.?:h 1 ox w ^l,h 2 ox w ^l} 
- V 



(2™ - wT 



V. 



(52) 



In the following, we will evaluate V. Define a, (3, 7 by 



(61) 



a 


A 


#{* 


G [l,n] 


■ "» 


= i,/4 2) 


= 0,Xi 


= 1} 


(53) 




A 


#{* 


G [l,n] 


: ^ 


= 0,^ 


— 1; %i 


= 1} 


(54) 


7 


A 


#{* 


G [l,n] 


: ^ 


= i,^ 2) 


— I; %i 


= 1}, 


(55) 



where hi 

(M 2) ,4 2) 



■ , /in 



ah 



(2) 

ensemble 7^2 4 consists of matrices of the form 



hi ©h 2 

^3 
hi 

V h 3 © h 4 y 



chosen from F 2 with uniform probability. From Theorem [T] 
we have 

3 19 7 



and h 2 = 

t 2 ,...,/!,„ Assume that hi o x w ^ 1 and 
h 2 o x w ^ 1. In this case, the equality (hi © h 2 ) o = 1 
holds if and only if the following two conditions hold: 
(i) {a, /3) = (1,0) or (a, f3) = (0,1), (ii) 2 < 7 < w. 
Suppose the case (a, /?) = (1,0). The number of possible 
pairs (hi,h 2 ) satisfying hi o x w ^ l,h 2 o x w ^ 1, 
(hi © h 2 ) o a;^ = 1 is given by 

[)(w~l)^ (n ~ w) - (56) 

7=2 ' 



{^}l=x = {l. 



2' 16' 16 



(62) 



On the other hand, 7^2,4, which is the set of matrices of the 
form 

( h x \ 
h 2 

V h 4 y 

has the average SS weight distribution 



(63) 



;c^2,4) 4 = J 1 3 25 _9_ , 
\ J w i w= i I ' 2' 16' 16 I ' 



(64) 
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where this distribution is derived using Lemma [2] It can be 
observed that the average SS weights of the extended ensemble 
are smaller those that of the original ensemble in the cases 
w = 3 and 4. rj 
The argument used in the proof of Theorem Q] can be used 
to derive the average SS weight distribution of the extended 
constant row weight ensemble with L = 2, the details are 
summarized in the Appendix. Here we consider the following 
example. 

Example 5: Let n = 100, m = 50. The average SS weight 
distribution of the extended constant row weight ensembles 

c (2) 

with L = 2, S w 100 ' 50 ' r , can be evaluated using Lemma [T3l of 
the Appendix. Tables HI and [TT1 present the two cases r = 10 
(sparse matrix) and r = 50 (dense matrix), respectively. We 
can see that the improvement due to extension is very small for 
the case r = 10. On the other hand, a significant improvement 



can be observed for the case r = 50. 

TABLE I 

Comparison of a non-extended ensemble and an extended 



□ 











UI 






1 


0.515 


0.515 


2 


0.217 


0.217 


3 


0.107 


0.107 


4 


0.0726 


0.0721 


5 


0.0748 


0.0737 


6 


0.123 


0.119 


7 


0.322 


0.308 


8 


1.33 


1.24 


9 


8.20 


7.54 


10 


71.5 


64.6 


n = 


IUU,m = 


5U,r = 10 



TABLE II 

Comparison of a non-extended ensemble and an extended 
ensemble(constant row weight ensemble, dense case). 



w 




<~>w 




1 


8.88 x 10~ i4 


8.88 x 10" 


14 


2 


2.65 x 10~ 12 


2.65 x 10" 


12 


3 


7.43 x 10" 6 


8.32 x 10" 


9 


4 


2.23 x 10° 


4.18 x 10" 


3 


5 


1.87 x 10 4 


3.08 x 10 2 






n = 100,m = 


50, r = 50 





This example suggests that the advantages of redundant exten- 
sion are more significant when the original matrix is dense. 
This is one of the major reasons that we focus on redundant 
random ensembles in the present study. 



C. Redundant random ensemble: L > 2 

It becomes difficult to evaluate the number of extended 
parity check matrices giving a stopping set for a given weight 
when L > 2. Instead of deriving an exact expression, we here 
utilize upper and lower bounds on the number of such parity 
check matrices to study the average SS weight distribution of 
extended ensembles. 



1 ) Number of generator matrices with minimum distance 
greater than or equal to 2: Let G be a binary K x N 
matrix(l < K < N). The weight distribution of the code 
generated from G (as a generator matrix) is defined by 

A W (G)= ^ E I l m G = c] (65) 

for 1 < w < N, where denotes F 2 X \0. The average of the 
weight distribution (where expectation is taken over a given 
ensemble Q) is given by 



Eg[A w (G)} =£P(G0 E 



E 

Geg C£Z( N <™) meF* 



I[mG = c] (66) 



The minimum distance of G is given by 



A 



(G) = min{z e [l,n] : A Z (G) ± 0}. 



(68) 



In order to prove the upper and lower bounds on the 
number of certain parity check matrices, we can use the first 
and second moment method[7], which requires the first and 
second moments of a random variable. The following lemma 
is presented in the problem section of [6] (the proof is given 
in the Appendix). 

Lemma 8: The first and second moments of A W (G) with 
respect to TZk n are given by 



E nK JA w (G)} = (2* -1)2 



-JY 



N 



(69) 



and 



E nK JA w {Gf] = E n „ N [A w (G)} 2 + E nK JA w (G)} 



1 



N 



-N 



(70) 



respectively, for w S [l,n], rj 
The next lemma is the basis of the lower bound on the av- 
erage SS weight distribution for redundant random ensembles. 

Lemma 9: The number of matrices in 1Zk,n which have 
minimum distance greater than or equal to 2 has a lower bound 
given by 

#{G e n K (G) > 2} 

> max{2 KJV max{l - (2 K - 1)2~ N N, 0}, 2 KN - K }. (71) 

(Proof) Let D = #{G 6 1Z k ,n ■ d min {G) > 2}. We first 
prove D > 2 KN max{l - (2 K - 1)2~ N N, 0}. The number of 
matrices whose minimum distance is greater than or equal to 
2 can be written in the form 

D = #{Ge7e K ,/v:^i(G)=0} 
= #{GeK K , N :^i(G) <1} 

E / [^i( G )< 1 ] 

= 2 KN p (G)I[MG)<l] 



Geii f 



2 KN Pr[A l {G) < 1], 



(72) 
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where Pr[A\(G) < 1] is given by 

Pr[A 1 (G)<l}= P(G)I[Ai(G) < 1]. (73) 

GgTZk,n 

The Markov inequality implies 

Pr[Ax{G) > 1] < E nK N [Ax{G)\, (74) 

which is equivalent to 

Pr[Ax(G) < 1] > max{l - E nK N [Ai(G)], 0}. (75) 

Substituting this upper bound into Eq.(|72l and using Eq.(|69l>, 
we have 

D > 2 KN mzx{l-E nKN [A 1 (G)},0} 

= 2 KN max{l - (2 K - 1)2~ N N, 0}. (76) 

We then consider the inequality D > 2 KN ~ K . Suppose the 
case that every row of G 1 G 1Zk,n is of even weight. We call 
this condition the even weight condition. In such a case, no 
linear combination of rows of G 1 gives a vector of weight 1. 
The number of K x TV matrices satisfying the even weight 
condition is 2 K ( N ~ X ^ because there exist 2 N ~ 1 even weight 
vectors of length N, j-j 
The next lemma will be required to prove an upper bound 
on the average SS weight distribution of redundant random 
ensembles. 

Lemma 10: The number of matrices in IZk.n which have 
minimum distance greater than or equal to 2 has an upper 
bound given by 

#{GeHK,N:d min (G) > 2} 

1-N2~ N 



< 2 



KN 



(2 K - 1)N2- N + 1 -N2~ N ' 



(77) 



(Proof) For a non-negative integer-valued random variable X, 
the following inequality holds [7], 



Pr[X = 0] < 



E[X 2 ] -E[X]' 



E[X 2 } 

Considering A\(G) as a random variable, we obtain 

#{G G Kk .N • drain (G) > 2} 

= #{G G 1Zk,n '■ A\{G) = 0} 
= 2 KN Pr[A 1 (G) =0] 

~ 2 En K .„[MG) 2 ] ' ( } 

From Lemma |8] the first and the second moments of A%(G) 
are given by 



Ek k , n [MG)] = (2 K -1)N2- N , 
En K , N [MGf] = ((2 K -l)N2- N f 

+ (2 K - 1)N2- N (1 - ^2"^) . (80) 

Substituting these expressions into Eq.d78ll. we have the claim 



of the lemma. 



2) Upper and lower bounds on average SS weight distri- 
butions: We are now ready to prove the following upper and 
lower bounds on the average SS weight distribution. 

Theorem 2 (Upper and lower bounds on Sw 1 ): The 
average SS weight distribution of the redundant random 
ensemble satisfies the inequalities 



Ol'Sn.n > 



77.(1) 



Q max{A m/i ,2- m } 



1 - w2- 



(2 L - l)w2- w + l- io2- 
for 1 < w < n, where A is defined by 



A 



A = max{l - {2 L - l)2" w iy,0}. 



(81) 

m/L 

(82) 



(83) 



(Proof) From the definition of the average SS weight distribu- 
tion, S w m ' n can be expressed as 



1 



#{H G fcW : q H {x) = 0} 



2^0 {*{H'e1l ( t L } l : qH ( X ) = 0} 



m/L 



1 In 

2 mn \w 



(G) > 2}2 



L(n — w) 



m/L 



,(84) 

where, in the above, the second equality was obtained by 
using the symmetric property of the ensemble, and the third 
equality arises from the property that L sub-blocks can be 
chosen independently. The last equality holds because 

#{H'eK£l:q H ,(x)=0} 

- #{G G Tl L , w : d min (G) > 2}2 Un - w \ (85) 

Note that d m i n (G) > 2 means that no linear combination 
of row vectors of G (except for all zero coefficients) has 
weight 1 . Applying the inequality in Lemma [10] to Eq. 
we immediately obtain the upper bound, 

s n ™- n - 1 ( n 

w ~ 2 mn \w 



X 



#{G G K L ,w * drain (G) > 2}2 



L(n—w) 



m/L 



2 mn \w 



1 (n\ /„ ~,Ln\ m / L 



2 mn \w 
n 



(B x 2 1 



w 



B m/L 



where B is defined by 



B = 



1 - w2- 



(86) 



(87) 



□ 



(2 L - l)w2~ w + 1 - w2~ w ' 
On the other hand, applying the inequality in Lemma [9] to 
Eq.(l86b. the lower bound can be derived. Let D' = #{G G 
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1Zk,n '■ dminiG) > 2}. The lower bound in Lemma [9] is 
equivalent to 



D' > max{2 Lw A,2 Lw - L }, 



which leads to the lower bound 

n£">„ 1 I n 



S 



I c\L(n—w) 



D'2 



m/ L 



Q max{A m / L ,2- m }. 



(89) 



□ 

It is easy to check that the upper bound and the lower bound 
coincide with the average SS weight distribution of the non- 
extended ensemble S^ m " n given in Eq. (flZb if L = 1. 

Example 6: Consider the case n — 100, to = 50, L = 2. 
In this case, we can compute exact values of the average 
SS weight distribution due to Theorem Q] Table [Til] presents 
the exact values (Theorem [D together with the values of the 
upper and lower bound(Theorem[2} of the average SS weight 
distribution of 1Z 50 100 . For the cases w = 1 and 2, we can 
see that the values of the upper and lower bounds coincide 
and they give the exact values. rj 



TABLE III 



Average SS weight distribution S w 

LOWER BOUNDS. 



(2) 



: EXACT, UPPER AND 



w 


Exact 




Upper 




Lower 




1 


8.88 x 10" 


14 


8.88 x 10- 


14 


8.88 x 10" 


14 


2 


4.40 x 10" 


12 


4.40 x 10" 


12 


4.40 x 10" 


12 


3 


1.05 x 10" 


8 


1.07 x 10" 


6 


1.44 x 10" 


10 


4 


4.15 x 10" 


3 


1.17 x 10" 


1 


3.48 x 10" 


9 


5 


2.58 x 10 2 




1.20 x 10 3 




1.02 x 10 1 





'(L > 2) 



An exact ( non-trivial) expression for S u 

does not at present exist. Let Ql,w = #{G G TZl,w ■ 
d m in(G) > 2}. The source of the difficulty in deriving an 
exact expression comes from the difficulty in counting Ql, w 
precisely. However, if both L and w are small, we can obtain 
Ql,w by an exhaustive computer search. Table HVl presents the 
values of Ql,w for 1 < w, L < 5 which have been evaluated 
by such an exhaustive computer search. 

TABLE IV 

Values of Q l>w (The number of matrices with minimum distance 
greater than or equal to 2). 



L\w 


1 2 


3 


4 


5 


1 


1 2 


5 


12 


27 


2 


1 4 


19 


112 


619 


3 


1 8 


71 


792 


10683 


4 


1 16 


271 


5416 


140251 


5 


1 32 


1055 


38472 


1751067 



The following lemma gives exact value of S-, 
know the value of Ql,w 



R 



if we 



Lemma 11: The average SS weight distribution of lZm}n is 
given by 



(88) ^"=2^(:)k- 2 



L(n—w) 



(90) 



(Proof) The claim of the lemma has already been derived as 
Eq.®. n 
Example 7: Consider the case n = 100, to = 50, L = 5. 
Combining Lemma QT| and the result presented in Table [IV] 
we can derive the exact values of the average SS weight 
distribution for 1 < w, L < 5. These values are presented in 
Table [V] together with the corresponding values of the upper 
and lower bounds. rj 



TABLE V 

Average SS weight distribution S m 5< 

LOWER BOUNDS. 



EXACT, UPPER AND 



w 


Exact 




Upper 




Lower 




1 


8.88 x 10" 


14 


8.88 x 10- 


-14 


8.88 x 10" 


14 


2 


4.40 x 10~ 


12 


4.40 x 10" 


12 


4.40 x 10- 


12 


3 


1.94 x 10- 


10 


1.93 x 10" 


-8 


1.44 x 10" 


10 


4 


1.73 x 10" 


8 


1.12 x 10" 


4 


3.48 x 10" 


9 


5 


1.13 x 10" 


5 


3.89 x 10" 


1 


6.69 x 10" 


8 



There is a trade-off between the extension degree L and the 
average SS weight distribution. The decoding complexity of 
BP-based iterative decoding increases as L increases because 
the number of rows in the extended matrix (2 L — l)(m/L) is 
an exponentially increasing function of L. On the other hand, 
a large L tends to give a larger stopping distance. The next 
example demonstrates such a trade off relation. 

Example 8: Figure [2] presents the relation between L and 
the upper bound of the average SS weight distribution. The 
horizontal axis of Figf2]represents the weight w. The ensemble 
assumed here is the random ensemble with n = 100, to = 50, 
namely 7^5o,ioo- We can observe that the upper bound on the 
average SS weight distribution decreases as L increases for a 
fixed weight. rj 



No extensic 
L= 
L= 
L=2 
L=E 


5 — H— 

« 
5 

■ 
























/ 

/ 










/- 


/ 

































Weight w 

n = 100, to = 50 

Fig. 2. Upper bounds on the average SS weight distribution of redundant 
random ensembles. 

Example 9: Figure [3] shows the block error probability of 
three example ensembles with m = 50, n = 100: the random 
ensemble (matrix A) , the redundant random ensemble with 
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L = 2 (matrix B) and the redundant random ensemble with 
L = 5(matrix C). The channel is BEC and BP is used in the 
decoder. It is observed that the decoding performance of matrix 
C is the best among the three matrices. The reason for these 
differing performances can be seen with reference to Table |VI] 
This table presents the stopping distance of the three matrices 
and their multiplicity. The multiplicity is the number of the 
stopping sets with size equal to the stopping distance. These 
values have been computed by an exhaustive computer search. 
The matrix C has the largest stopping distance, 7, which 
gives a smaller block error probability compared with those of 
matrices A(stopping distance 4) and B(stopping distance 5). 




0.0001 I i i i i 1 

0.02 0.04 0.06 0.08 0.1 

Erasure probability 



n = 100, to = 50 

Fig. 3. Block error probabilities of examples of random and redundant 
random ensembles (Matrices A,B, and C from top to bottom). 

TABLE VI 

Stopping distance and multiplicity of matrices A,B and C. 







n = 100, m 


= 50 




Matrix 


L 


Stopping distance 


Multiplicity 


# of rows 


A 


1 


4 


1 


50 


B 


2 


5 


262 


75 


C 


5 


7 


1365 


310 



□ 



We here compare a high rate redundant random ensemble 
with constant row weight ensembles and bipartite ensembles 
in terms of their typical stopping distances. 

Example 10: Consider the case n = 1024, to = 32. We can 
show that the maximum value of the typical stopping distance 

of £-n,m,r ^ 

max S Cl024 ' 32 - r = 3. 

r6[l,1024] 

For bipartite ensembles, we have 

max-5 Bc32c = 3. 

c>3 

These results mean that there are no constant row weight 
ensembles and bipartite ensembles with n = 1024, to = 32 
which achieve the typical stopping distance 4. On the other 
hand, the redundant random ensemble (n — 1024, to = 
32, L = 8) has a larger typical stopping distance: 

(5^32,1024 = 4. 

In this case, the extended random ensemble is expected to give 
asymptotically (i.e., in the regime of small erasure probability) 
better decoding performance (with BP) than the constant 
row weight ensemble with any row weight and the bipartite 
ensemble. 

Figure [4] presents the block error probabilities of examples 
of a redundant random ensemble (n = 1024, to = 32, L = 
4, 8) and a constant row weight ensemble (n — 1024, to = 
32, r = 100, 200, 300). Note that the size of the parity check 
matrices used in the BP decoder is 120 x 1024 (redundant 
random, L = 4), 1020 x 1024 (redundant random, L = 8), 32 x 
1024 (constant row weight), respectively. It may be observed 
that the example redundant random ensembles give steeper 
error curves than those of the example constant row weight 
ensembles. This difference in decoding performance could be 
explained from the the difference in typical stopping distance 
discussed above. r-\ 



D. Typical stopping distance 

From the average SS weight distribution, we can retrieve 
some information about the stopping distance of matrices 
contained in an ensemble. 

Definition 7 (Typical stopping distance): The typical stop- 
ping distance of an ensemble Q is defined by 

S G =min|.s 6 [l,n] : ^ S£ > lj . (91) 

The condition Y^w=i Sw(H) = is equivalent to A(iJ) > 
s. It is evident that there exists a matrix H G Q satisfying 

Yf w Ji S w (H) = 0, because the average J2t=i $w is strictly 
smaller than 1. This means that there exists a matrix with a 
stopping distance larger than or equal to the typical stopping 
distance 5 g . 



0.1 



C 01 



C01 



0.0001 





M ;■—■-»■ ' ' ■ 


X'" / // 




// 1 






Redundant random(L=4) — + — 
Redundant random(L=8) — % — 
Constant row weight(r=i oo) * 




Constant row weight( r=2001 Q 
Constant row weight(r=300) - -» — 



0.001 0.01 0.1 



Erasure probability 

n = 1000, to = 32 

Fig. 4. Block eiTor probabilities (BP on BEC) of examples of a redundant 
random ensemble and constant row weight ensemble. 
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IV. Asymptotic growth rate of the average SS 

WEIGHT DISTRIBUTIONS OF REDUNDANT RANDOM 
ENSEMBLES 

In this section, we will discuss the asymptotic {i.e., in the 
limit as n goes to infinity) behavior of the average SS weight 
distribution. 

A. Bounds on asymptotic growth rate 

We will consider the asymptotic behavior of the average SS 
weight distribution of the redundant random ensembles. 
The asymptotic growth rate erg (R, /z) is defined by 



A 



1 



TV: 



,{R,fi) = Jim ^ log S tn ll - R)n " 



(92) 



n— voc TL 

for < £ < 1,0 < fi < 1 — R. The parameter p, is 
called normalized extension degree. It is evident that, from 
the definition of redundant extension, the above definition of 
<jp(R,fi) is well defined only if (1 — R)/fi is an integer. 

The next corollary gives a lower bound on at(R, fi). 

Corollary 1: The asymptotic growth rate at(R, fi) can be 
lower bounded by 

(Proof) We first consider the case I < fi. From Theorem [2] 
we have the following inequality, 



~-> {tin) 



>max{A'( 1 -^^,2-( 1 - ii )"} 



where 



A' = max{l - (2" n - l)2~^n, 0} 



(94) 



(95) 



It is evident that 1 - (2^ n - l)2^ ln £n -> -oo in the limit as 
n goes to infinity. This implies the equality 

maxi/ 1 -^/", = (96) 

holds for sufficiently large n. Upon using this result, we 
immediately obtain a lower bound, 

a t (R,n) = Urn - log slf~ R ^ n 

n— *oo 77, 

> lim -log2-( 1 -«)'Y" 



- (1 - R). 



(97) 



We next consider the case £ > p,. In this case, 1 — (2 M ™ — 
l)2~ ln £n — > 1 in the limit as n goes to infinity, and so 

max{A /(1 - R)/ ' 1 , 2- (1 - fl) "} -► 1 (98) 
in the limit as n —> oo. Upon using this result, we obtain 

a e (R,p) = lim -logSlf- R ^' n 



> lim - lo Q 



= H(£). 

The next corollary provides an upper bound on ai (R, (i) 



(99) 
□ 



Corollary 2: The asymptotic growth rate a^(R,fi) has an 
upper bound given by 



H{£)-{l-R)(l-l), £ 



< 



H{£), 



£> fi. 



(100) 



(Proof) The upper bound in Theorem [2] can be rewritten in the 
form 



< 



< 



1 - in2~ 



2^ n £n2- hl + 1 - 2£n2- £n 
l-£n2-^\ {1 - R) ^ (n 



2^- t > n £n 



hi 



(101) 



for sufficiently large n. The last inequality holds because 1 — 
2£n2~ £n is always positive for large n. Thus, the asymptotic 
growth rate can be bounded from above: 

a e (R, /i) < H(£) + 0— ^ lim - log 1 ~ ' " 2 ' " 



= ff(i) 



n— >oo n 2^~^ n £n 



x lim - flog(l - £n2- ln ) - log(2 ( ^ £)n ) - \og{£n) 



= H {£) + ^ — lim -{-U-£) n ) 

n~*oo n 

= H(£)-{1-R) (l-- 

On the other hand, the inequality 

1 - £n2- in 

< 1 



(102) 



_ i)g n 2-^ + 1 - £n2~ 



tn 



(103) 



leads to another (trivial) upper bound on S e 



b ^ ~ [in 



(104) 



The asymptotic form of this upper bound is given by 

<r t {R,ti) < H{£). (105) 



If £ < fi, the upper bound ( 11021 i gives smaller values than the 
trivial bound ( 1105b . If I > fi, the trivial bound ( 1105b becomes 
tighter. When £ — fi, both of the bounds yield the same value 
H(£). ' n 

Combining the above two corollaries, we can see that 
(Je(R, fi) — H(£) for £ > fi. That is, the upper and lower 
bounds are asymptotically tight when £ > fi. 

Example 11: Figure [5] shows the lower bound (Corollary Q} 
and the upper bound (Corollary |2]i for the case R = 0.5, fi = 
0.25. The horizontal axis of Fig|5] represents the normalized 
weight £. The curve H{£) (the asymptotic growth rate of non- 
extended ensemble lZ(i-R) n . n ) is also included in Fig|5]as a 
reference. 
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Redundant random ensemble R=0.5 
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0.065 








































0-027 











Normalized extension degree it 



Fig. 5. Upper and lower bounds on the asymptotic growth rate of the Fig. 6. Upper bounds on the critical exponent of redundant random ensembles 
redundant random ensemble (Ft = 0.5, = 0.25). (R = 0.5,0.75). 



B. Critical exponent 

The critical exponent of an ensemble is the normalized 
weight such that the asymptotic growth rate changes from 
negative to positive. The explicit definition of the critical 
exponent of a redundant random ensemble is given below: 

Definition 8 (Critical exponent): The critical exponent of 
the redundant random ensemble is defined by 



A 



a(R,n) = inf{£ > : a t {R,p) > 0}. 



(106) 



□ 



The following lemma, which gives bounds on the critical 
exponent, is a direct consequence of Corollaries Q] and [2] 
Lemma 12: The following inequality holds 

a L {R,p) <a(R,p) < au {R,fi), (107) 

where cxl(R, p.) is the minimum positive solution of H(£) — 
(1 — R)(l — i/p) = 0, and ajj(R : p) is the minimum positive 
solution of H(£) - (1 - R) = 0. 

(Proof) From Corollaries Q] and [2] it is evident that the claim 



holds. 



□ 



The critical exponent of the bipartite ensemble is given by[2], 



A 



/3(c, d) = ml{i > : y e (c, d)>0}. 



(108) 



Figure|6]presents the lower bound on the critical exponent of 
the redundant random ensemble with R = 0.5 and R = 0.75. 
The horizontal axis is the normalized extension degree p. Of 
course, if (1 — R)/p is not an integer, the lower bound is 
not well defined. However, for simplicity, the lower bound is 
plotted as if it were valid in the entire range < p < 1 — R. 
We can see that the exponent increases as p increases. Since 
the parameter p can be considered as a measure of decoding 
complexity, the plots in Fig. [6] can be regarded as the trade- 
off curves between decoding complexity and the asymptotic 
decoding performance. 

We then compare the critical exponent of the redundant 
random ensemble and the bipartite ensemble with a design 
rate of 0.5. It is known that there exists an optimal choice 
of the variable node degree c to attain the maximum critical 
exponent (3(c, d). The best value is max c >3 /3(c, 2c) = 0.065, 



which is obtained when c = 7, d = 14. In other words, no 
bipartite ensemble with R = 0.5 yields a critical exponent 
larger than 0.065. Note that the plot of the lower bound on 
the critical exponent of the redundant random ensemble takes 
larger values than 0.065 if /i is sufficiently large. This result 
implies that the asymptotic BP-performance on BEC of a 
dense matrix can be better than that of a sparse matrix. 

In the case of a high code rate (design rate R = 0.75), we 
have max c /3(c, 4c) = 0.027. The maximum value is obtained 
when c = 9, d = 36. We can observe that, as for the former 
case, the redundant random ensemble with sufficiently large 
p gives larger values. 

Example 12: Figure [7] presents the asymptotic growth rate 
of the random ensemble (R = 0.5), bipartite ensemble (c = 
7,d= 14), constant row weight ensemble [R = 0.5, r = 15) 
and the redundant random ensemble (R = 0.5, p = 0.5, upper 
bound). The parameters of the bipartite ensemble and the 
constant row weight ensemble are chosen so as to maximize 
the critical exponent under the constraint R = 0.5. In this 
case, both the bipartite and the constant row weight ensembles 
have almost the same maximum critical exponent of 0.065. On 
the other hand, we have (0.5, 0.5) = 0.083, which is larger 
than the maximum critical exponent of the constant row weight 
ensemble and the bipartite ensemble. rj 





Constant row 


Random (R=0.5) 
Bipartite (C=7,cM4j 
weight (R r. 5. r= 15 






Redundant Random(upper] (R=0.5,mu = 


0.5) 






























/ 

















































0.15 

Normalized weight 



Fig. 7. 
constant 
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V. Conclusion 

In this paper, the average SS weight distribution and the 
asymptotic growth rate of redundant random ensembles have 
been analyzed. The results obtained in the paper describe 
one aspect of the trade-off between decoding complexity of 
BP(extension degree) and decoding performance. It is shown 
that, in some cases, a dense matrix with linearly dependent 
rows can yield a better decoding performance over BEC 
than a regular LDPC matrix with comparable parameters. In 
particular, in the high rate regime, a redundant matrix appears 
to offer promising performance not only for BEC, but also for 
other channels. It is hoped that further research concerning 
this result can be undertaken. 
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Appendix 

Proof of the first and second moment of A w (G) 

From a simple counting argument, we obtain #{G G 
1Zk,n '■ rnG — c] = 2^ K ^^ N . This equation leads to the 
transformation 

En K , N [A w {G)] = 



((2 K - l) 2 - (2 K - 1)) 2- 2N 



+ (2 K - 1)2^ 

= {En K , N [A w {G)\y 



+ Ek KiN [A w (G)}[1-[ w 12 



N 



-N 



(111) 



Redundant constant row weight ensemble: L = 2 

As in the case of the redundant random ensemble, it is 
possible to write down a simple formula for the redundant 
constant row weight ensembles if L = 2. 

Lemma 13: The average SS weight distribution of the 
redundant constant row weight ensemble with parameters 
n, m(l < m < n),r(r > 1) and L = 2 is given by 

2 \ m/2 



S u 



where V is given by 

min{-u; — l,r — 1} 




(112) 



v.* g (»-,,(;) 



2 KN 

x zZ zZ #{Gen K ,N :mG = c} 
ceZ (jv,u,) meFf 



w\ I n — w \ / n — w 

(113) 

(Proof) The proof of the theorem is almost same as the proof 
of Theorem Q] Let 

U = #{{hiM) G Z (n ^ x Z M :h l0 x w ?l, 

h 2 °x w ^l,(h 1 ®h 2 )°x^l}. (114) 

and 



2 KN 



£ #{G G Tl K , N :mG = c} V = #{{hiM) € x : h, o x + 1, 



meFi 



( N ) 



h 2 ox ^ 1, (hi ®h 2 ) ox = 1} (115) 

A simple combinatorial argument similar to that used in the 
proof of Theorem Q] leads to a simple formula for V, 



N 



(2 K - 1)2 



— N 



(109) 



We next consider the second moment. The second moment 
can be written as 

E-Rk.n l(A w (G)) 2 ] 

= I[miG = ci,m 2 G = c 2 ] 

= zZ 7 t mi ^ m 2l x i 

c x ,c 2 m u m 2 

c u c 2 m 1 ,m 2 

+ J2 I[m l =m 2 ]I[c 1 = c 2 ]Xz (110) 

d,c 2 m 1 .yn 2 

From a combinatorial argument, we have Xi = 2~ 2N ,X 2 = 
0, X 3 = 2~ N . Finally, we get 

E n K , N 1{A W (G)) 2 ] 



min{w— t,r— 1} 

v = 2 £ («, - 7 ) 

7=2 



uA/ n — w \ I n — 

l) \r -7 - 1/ \ r - 7, 

(116) 



Upon using the relation 

17 = #{(h u h 2 ) G 2 (n ' r) xZ( ,l ' r ) : h x ox ± l,h 2 ox f 1}-V, 

(117) 



we have 



v-u" r )-i - ::?n -<■ m, s , 



for 0<w<n,r>2. The average SS weight distribution is 
therefore given by 

wC n ,m,r j; eZ (»,») 

= j*ynif-{H G C^r ; g^lj = 0} 



U/ rrm/2 

(;) m ' 



(119) 
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where the last inequality coincides with the claim of the 
lemma. rj 
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